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Face Swapping Video 
Detection with CNN 


e Deepfakes video 


e A simple and effective CNN 
e Face recognition based method 


Deepfakes Video 


When Face Recognition 
Systems Meet Deepfakes 


Vulnerable face comparison before fake 
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Face Swapping Video 
Generation 


Characteristics 


Swap victim’s face in every frame independently 
Not End2End 

Only manipulate central face area 

Autoencoder 


DeepFakes 


Faces Swapping 


Extracting video frames Generating the fake video 


Deepfakes Training Phase 
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Deepfakes Generation Phase 


e Convert 
— Person A Encoder -> Person B Decoder 


e Merge back 
— Gaussian Blur/Color Average 
— Poisson Image Editing 


Person B 


Compressed 
Real image representation Fake image 


Face Swapping Video 
Detection with CNN 


e Deepfakes video 
e Å simple and effective CNN 


— capturing low-level features of the images 


e Face recognition based method 


A Simple and Effective CNN 


Design purpose 
e Input contains marginal(background) information. 
e Capture low-level features of the images. 
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A Simple and Effective CNN 


Training 
e Dataset from VidTIMIT 
— 67600 fake faces and 66629 real faces 
— low quality and high quality images 
e Cropped faces 
— with face landmark detector MTCNN 
— obtain 1.5 scaled bounding box 
e Augmented data 
— horizontal flipping 
— randomly zooming 
— shearing transformations 


A Simple and Effective CNN 


Characteristics 
e 3 convolution layers 
e Accuracy rate: 99% 


Face Swapping Video 
Detection with CNN 
e Deepfakes video 
*« A simple and effective CNN 


e Face recognition based method 
— capturing high-level features of faces 


What is FaceNet? 


Characteristics 
e SOTA CNN for face recognition 
e Model structure 
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FaceNet: A unified embedding for face recognition and clustering 


A FaceNet based SVM 
Classifier 


abandon 


Training 
e Central face area 


— No margin/background 
— Onlyfacearea 


e Dataset from VidTIMIT 


FaceNet 


Real Faces 
Dataset 


Feature Extraction 


A FaceNet based SVM 


Classifier 


Characteristics 

e FaceNet used for extracting face features 
e SVM for binary classification 

e Accuracy rate: 94% 


Features Extraction 


Face Detection Binary Classification 


A Simple and Effective CNN 


Accuracy rate: 99% 


A FaceNet based SVM 
Glassiflex 


summary 


e CNN for image classification 
— Å simple architecture can work well. 
— catching low-level features: contours, edges... 


» A FaceNet based SVM classifier 


— using FR to catch features of fake faces 
— using SVM for binary classification 


— 64% accuracy rate for the misclassification set from the 
simple CNN based classifier 
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